DGROUP.TXT version 2.0 Copyright(C) 1993, Robert G. Montgomery. All Rights Reserved. The following is a discussion of DGROUP, and its relationship to the following error messages: Unrecoverable Error 650 Processor Stack Fault (UE 650) Unrecoverable Error 667 Eval Stack Fault (UE 667) Unrecoverable Error 668 Eval Stack Fault (UE 668) Unrecoverable Error 669 Eval Stack Fault (UE 669) The following discussion is based upon my understanding what is going on with DGROUP, as it relates to programming with CA-Clipper (hereafter referred to as Clipper), version 5.0 through 5.2c, and probably later. This is based mostly on information obtained through messages on the Clipper forum on CIS. I am not an authority on this, just a common user of Clipper who has figured a way around a problem I was having with Clipper. I furnish the following as a favor to all Clipper users, and I furnish the following information without any guarantee that this is the way it is, and therefore cannot accept any responsibility for anything which may come of it. All I know is that it works for me. Please use at your own risk, and please don't sue me . If anyone has anything to add, or wishes to point out where I might be downright wrong on something discussed here, feel free to contact me through CIS at 75660,31. ***************************************************************************** * DGROUP * ***************************************************************************** DGROUP is an area of memory which exists in large model programs (like our Clipper apps). This is where many Clipper internals are stored during the execution of your programs. DGROUP structure: direction of growth <---|---> <---| +---------------------+-----------+------------+---------+--------------+ | C & ASM static data | CPU stack | Eval Stack | "space" | Memvar Table | +---------------------+-----------+------------+---------+--------------+ | allocated at link-time | allocated at run-time | |<------ DS Avail from //INFO ------->| |<---------------------------- total of 64K --------------------------->| DGROUP is the area of memory where the DS (Data Segment) and SS (Stack Segment) registers of the CPU point to during the execution of large model programs. This way, all pointers to static data during execution of a program can be done with near pointers, which are faster since they are only 16 bit pointers. By assuming SS=DS, access to C and ASM local variables may also happen with near pointers. And by putting the Eval Stack and Memvar Table in DGROUP, they too can be accessed with near pointers. This is all in the interest of speed. Some background information. What is a VALUE? A VALUE is a data type defined internally to Clipper which is a 14 byte block of memory stored in DGROUP which contains the information necessary to define a Clipper value. Stored within the VALUE is the type of the Clipper value (Character, Numeric, etc.), and the value itself, if possible. If the value is too large to be stored within the 14 byte VALUE record (i.e. character strings, arrays, etc.), a VM segment handle is stored within the VALUE, and the actual value is stored in a VM segment somewhere outside of DGROUP. Both the Eval Stack and the Memvar Table are arrays of 14 byte VALUE records. What is an ITEM? An ITEM is a near pointer to a VALUE. ITEMs are manipulated using ITEM.API, from C & ASM code. When an ITEM is created, a VALUE is allocated in the Memvar Table, and a near pointer to the 14 byte VALUE record is returned by the _item*() function. What is the purpose of the different areas within DGROUP? C & ASM static This is where static variables, and literal strings, defined within C or Assembler code, are stored. Many third party libraries, as well as many parts of the Clipper run-time, are written in C and/or assembler and use up DGROUP in this area. The amount of its allocation is depends on the variables and literal strings defined. You cannot do much about this except for not linking in code which eats it up, or modifying the C and/or ASM code which eats it up. The amount of C & ASM may be determined with following formula, run //INFO to get the DS Avail value. = 64K - - Or look in map file generated by the linker, and total up everything in DGROUP except for the stack. If you are using Exospace, the //info switch does not work so you must look at the map file to determine this. CPU Stack This is where procedure/function return addresses are stored, as well as local variables defined within C or Assembler code. The amount of its allocation is dependent on the /STACK or SET PROCEDURE DEPTH parameters given to the linker. Its default is 4096 bytes, but could be changed by the inclusion of .OBJ or .LIB files which request that it increase. As long as you don't get UE 650 processor stack fault, or system hangups, you can reduce the size of the CPU stack to save DGROUP memory for other things. Usually takes 4096 bytes, but may be adjusted by linker. Eval Stack This is where VALUEs for local and private variables and parameters are stored. Note that only the VALUEs associated with the variables are stored here, private variables also eat up space in the symbol table, locals do not. Memvar Table This is where clipper static variables are stored, as well as the VALUES which are associated with the ITEMs which are allocated through ITEM.API. It is a heap, and if all slots within the heap are filled when a call to _itemNew() is made, it expands the static area to allow for allocation of more ITEMs. ITEMs are allocated internally by Clipper, as well as by C & ASM routines, including many third party libraries. "space" This is the area in which the Eval Stack and Memvar Table expand into. Hopefully, conventional memory is not in short supply, but if it is, then some or all of the "space" may be reallocated to the "Conventional Memory Pool". If this happens, and then there is an attempt to expand either the Eval Stack or Memvar Table into the reallocated "space", you will get UE 668 or UE 669, Eval Stack Fault. The functions fix_UE668() and/or fix_UE669() will allow you to push the boundaries of the "space" inward toward the center of the "space", reducing your chances of getting the errors. What determines the boundaries between the 5 sections of DGROUP? The boundary between the C & ASM static data area and the CPU Stack, as well as the boundary between the CPU Stack and the Eval Stack, are set by the linker and remain fixed throughout the execution of your application. The Eval Stack expands upward in memory, and the boundary between the Eval Stack and the "space" is set by a "watermark" which marks the highest extent in which the Eval Stack has been expanded throughout the execution of the application. The Memvar Table expands downward in memory, and the boundary between the Memvar Table and the "space" is set by a "watermark" which marks the lowest extent in which the Memvar Table has been expanded. The memory between these "watermarks" is the "space". Under low memory situations, the VM will reallocate the "space" to the conventional memory pool. Determination of the position of the "watermarks" may only be done through access to UNDOCUMENTED public variables defined in CLIPPER.LIB, and they may only be accessed from C and/or ASM code. Specifically, the variable _evalhigh (near pointer to void) is the high "watermark" for the Eval Stack, and the variable _estatlow (near pointer to void) is the low "watermark" for the Memvar Table. In C, it is useful to type-cast their pointer values to unsigned integers (USHORT), and pass their integer values back to Clipper for the debugging programmer to see. It is useful to consider the values of these pointer variables when debugging programs, especially those which access ITEM.API, or use third party libraries which access ITEM.API. Clipper callable functions for determination of these pointer values, as well as other useful functions for determination of the state of DGROUP, are included in the file DGROUP.ZIP in lib 2 in the CA-Clipper forum on Compuserve. Writing of routines which access these pointer values is recommended only for debugging purposes, since they may be changed in upcoming releases of Clipper. ***************************************************************************** * Unrecoverable Error 650 * ***************************************************************************** Unrecoverable Error 650 - Processor Stack Overflow / Out of Stack Space. Your program has exhausted the CPU Stack memory. The most straightforward cause is having function and procedure calls nested too deep for the CPU stack to hold all of the required activation records for the called procedures and functions. This may be fixed by using the linker STACK command (or SET PROCEDURE DEPTH command) to expand the CPU stack for an application. Note that any memory added to the CPU stack gets taken from the Eval Stack and Memvar Table, so expand the stack carefully in small increments, until you do not get UE 650 anymore. Another cause is infinite recursion. This is the case if expanding the CPU Stack with a linker command does not work, since infinite recursion eats up all of the CPU Stack regardless of the size of it. Note that when running under Clipper, when an error is generated causing a call to the error system, and there is an error in the error system, another call to the error system will happen, causing another error to be generated, causing another call to the error system, and so on ... infinite recursion. One way to check for this is by viewing the call stack from within the Clipper debugger, and looking for repeated procedure/function names. ***************************************************************************** * Unrecoverable Error 667 * ***************************************************************************** Unrecoverable Error 667 - Eval Stack Fault This is caused by Eval Stack and/or Memvar Table Overflow, which means that the Eval Stack and the Memvar Table have overrun each other. One cause is not having enough memory in DGROUP (see DGROUP explanation for more info on this) available after C & ASM static data and the CPU Stack have been allocated. One way to check for this is by running your application with the //info command line switch and looking at the DS Avail value. Another way is to look in the map file generated by your linker and adding up the lengths of all segments which are linked into DGROUP. Both methods yield the same result, but the //info switch does not work with Exospace'd applications. If the value is under 20K, this may be your problem, and if it is below 10K, this is almost definitely your problem. If this is your problem, the only way to fix the problem is to not link in as much C & ASM static data, and/or reducing the size of the CPU Stack. Another cause is too many local, static, and private variables defined at one time. Note that ITEMs allocated through ITEM.API are basically the same as static variables, except that they can be created and destroyed. Static variables cannot be destroyed. Local variables are created when a function is called and destroyed when a function returns. Note that some Clipper internals create and destroy them as well. The total number of variables which may be instantiated at one time is roughly equal to the DS Avail value (see above) divided by 14. Note that it is possible (but improbable) that this error may be generated by infinite recursion, where the program runs out of Eval Stack before it runs out of CPU Stack. Also note that it is possible to get infinite recursion by having an error in your error handling system, where an error will generate a call to the error system, where another error generates another call, and so on until the program runs out of Eval Stack. See the section on DGROUP for more information. ***************************************************************************** * Unrecoverable Error 668 * ***************************************************************************** Unrecoverable Error 668 - Eval Stack Fault This is caused by Eval Stack Overflow into Locked VM Segment. Your program attempted to expand the Eval Stack into an area of DGROUP which contains a locked VM segment. Under low memory situations, the Clipper VM will reallocate memory in DGROUP between the Eval Stack and the Memvar Table to the conventional memory pool. After this happens, if your program needs to expand the Eval Stack into the reallocated memory, and that memory contains a locked VM segment, you get this error. Note that it is possible (but improbable) that this error may be generated by infinite recursion, where the program runs out of Eval Stack before it runs out of CPU Stack. Also note that it is possible to get infinite recursion by having an error in your error handling system, where an error will generate a call to the error system, where another error generates another call, and so on until the program runs out of Eval Stack. When running under Protected Mode using a DOS Extender (i.e. Exospace), it is very unlikely that this error will happen since there is plenty of memory available, so that the VM will not need to reallocate memory from DGROUP. See the section on DGROUP for more information. Also, see DGROUP.ZIP in lib 2 of the CA-Clipper forum on Compuserve, which contains the Clipper source code for fix_UE668(), a function which allows you to coerce the VMM into leaving some more memory for the Eval Stack when it reallocates DGROUP. Fix_UE668() is a Clipper function which creates a whole mess of local variables and returns, which will move the "watermark" (see DGROUP) that marks the highest expansion of the Eval Stack. It should be called in one of the first statements in your program. If the number of local variables declared and then released by Fix_UE668() is more than the total which will be needed during the execution of your program, you will not get UE 668 anymore. ***************************************************************************** * Unrecoverable Error 669 * ***************************************************************************** Unrecoverable Error 669 - Eval Stack Fault This is caused by Memvar Table Overflow into Locked VM Segment. Your program attempted to expand the Memvar Table into an area of DGROUP which contains a locked VM segment. Under low memory situations, the Clipper VM will reallocate memory in DGROUP between the Eval Stack and the Memvar Table to the conventional memory pool. After this happens, if your program needs to expand the Memvar Table into the reallocated memory, and that memory contains a locked VM segment, you get this error. Note that it is possible (but improbable) that this error may be generated by infinite recursion, where the program runs out of Memvar Table space before it runs out of CPU Stack. Also note that it is possible to get infinite recursion by having an error in your error handling system, where an error will generate a call to the error system, where another error generates another call, and so on until the program runs out of Memvar Table space. When running under Protected Mode using a DOS Extender (i.e. Exospace), it is very unlikely that this error will happen since there is plenty of memory available, so that the VM will not need to reallocate memory from DGROUP. See the section on DGROUP for more information. Also, see DGROUP.ZIP in lib 2 of the CA-Clipper forum on Compuserve, which contains the C source code for fix_UE669(), a function which allows you to coerce the VMM into leaving some more memory for the Memvar Table. Fix_UE669() is a C function which creates a whole mess of ITEMs through ITEM.API, releases them, and returns. This will move the "watermark" (see DGROUP) which marks the lowest expansion of the Memvar Table. It should be called in one of the first statements in your program. If the number of ITEMs allocated and then released through fix_UE669() is more than the total which will be needed during the execution of your program, you will not get UE 669 anymore.